Word Alignment for Languages with Scarce Resources

This paper presents the task definition, resources, participating systems, and comparative results for the shared task on word alignment, which was organized as part of the ACL 2005 Workshop on Building and Using Parallel Texts. The shared task included English–Inuktitut, Romanian–English, and English–Hindi sub-tasks, and drew the participation of ten teams from around the world with a total of 50 systems. 1 Defining a Word Alignment Shared Task The task of word alignment consists of finding correspondences between words and phrases in parallel texts. Assuming a sentence aligned bilingual corpus in languages L1 and L2, the task of a word alignment system is to indicate which word token in the corpus of language L1 corresponds to which word token in the corpus of language L2. This year’s shared task follows on the success of the previous word alignment evaluation that was organized during the HLT/NAACL 2003 workshop on ”Building and Using Parallel Texts: Data Driven Machine Translation and Beyond” (Mihalcea and Pedersen, 2003). However, the current edition is distinct in that it has a focus on languages with scarce resources. Participating teams were provided with training and test data for three language pairs, accounting for different levels of data scarceness: (1) English–Inuktitut (2 million words training data), (2) Romanian–English (1 million words), and (3) English–Hindi (60,000 words). Similar to the previous word alignment evaluation and with the Machine Translation evaluation exercises organized by NIST, two different subtasks were defined: (1) Limited resources, where systems were allowed to use only the resources provided. (2) Unlimited resources, where systems were allowed to use any resources in addition to those provided. Such resources had to be explicitly mentioned in the system description. Test data were released one week prior to the deadline for result submissions. Participating teams were asked to produce word alignments, following a common format as specified below, and submit their output by a certain deadline. Results were returned to each team within three days of submission. 1.1 Word Alignment Output Format The word alignment result files had to include one line for each word-to-word alignment. Additionally, they had to follow the format specified in Figure 1. Note that the and confidence fields overlap in their meaning. The intent of having both fields available was to enable participating teams to draw their own line on what they considered to be a Sure or Probable alignment. Both these fields were optional, with some standard values assigned by default. 1.1.1 A Running Word Alignment Example Consider the following two aligned sentences: [English] s snum=18 They had gone . /s [French] s snum=18 Ils étaient allés . /s A correct word alignment for this sentence is: 18 1 1 18 2 2 18 3 3 18 4 4